Light emitting diode flicker mitigation

ABSTRACT

Systems and methods are provided for detecting a flashing light on one or more traffic signal devices. The method includes capturing a series of images of one or more traffic signal elements in a traffic signal device over a length of time. The method further includes, for each traffic signal element, analyzing the series of images to determine one or more time periods at which the traffic signal element is in an on state or an off state, and analyzing the time periods to determine one or more distinct on states and one or more distinct off states. The method further includes identifying one or more cycles correlating to a distinct on state immediately followed by a distinct off state, or a distinct off state immediately followed by a distinct on state, and, upon identifying a threshold number adjacent cycles, classifying the traffic signal element as a flashing light.

BACKGROUND Statement of the Technical Field

The present disclosure relates to traffic signal element stateclassification and, in particular, to mitigating light emitting diode(“LED”) flickering during traffic signal element state classification.

Description of the Related Art

Object detection and analyzation are critical to safe driving,particularly pertaining to autonomous vehicles (“AVs”). One type ofobject that requires particular care when identifying and analyzing is atraffic signal device. Traffic signal devices indicate when it is thesafe, legal, and appropriate time for vehicles to pass or enter certainintersections or other regions. For this reason, AVs require the abilityto accurately detect and analyze traffic signal devices.

Traffic signal devices may be detected using standard object detectiontechniques, and image analysis is performed for each traffic signalelement in the traffic signal device in order to determine a state foreach traffic signal element in the traffic signal device. However,modern traffic signal devices often include light emitting diodes(“LEDs”) for the traffic signal elements. LEDs have irregular variationsin their intensities, which vary according to power cycles. When thesetraffic signal devices are captured in video sequences, they may appearto flicker due to the variations in their intensities and the frequencyand exposure settings of the camera used. This flickering poses aproblem for traffic signal element state determination since thisflickering could potentially be misinterpreted for a flashing trafficsignal element (e.g., a flashing red light or a flashing yellow light).

Therefore, for at least these reasons, a better method of accuratelydetermining between flashing traffic signal elements and traffic signalelements which are flickering is needed.

SUMMARY

According to an aspect of the present disclosure, a method of detectinga flashing light on one or more traffic signal devices is provided. Themethod includes, by one or more image capturing devices of an autonomousvehicle, capturing a series of images of one or more traffic signalelements over a length of time, wherein each traffic signal element isin a traffic signal device. The method further includes, by anautonomous vehicle control system of the autonomous vehicle, for eachtraffic signal element in the one or more traffic signal elements,analyzing the series of images to determine one or more time periods atwhich the traffic signal element is in an on state, and one or more timeperiods at which the traffic signal element is in an off state, andanalyzing the time periods to determine one or more distinct on statesand one or more distinct off states. A distinct on state correlates to atime period when the traffic signal element is in the on state for alength of time that equals to or exceeds a lower threshold, and adistinct off state correlates to a time period when the traffic signalelement is in the off state for a length of time that equals to orexceeds a lower threshold. The method further includes identifying oneor more cycles correlating to a distinct on state immediately followedby a distinct off state, or a distinct off state immediately followed bya distinct on state, and, upon identifying a threshold number adjacentcycles, classifying the traffic signal element as a flashing light.

According to various embodiments, the analyzing the series of imagesincludes generating a confidence score, for each image in the series ofimages, that the traffic signal element is in the on state.

According to various embodiments, the analyzing the series of imagesincludes generating a confidence score, for each image in the series ofimages, that the traffic signal element is in the off state.

According to various embodiments, the method further includes, by theautonomous vehicle control system of the autonomous vehicle, sending acommand to the autonomous vehicle to perform one or more instructionsassociated with the flashing light.

According to various embodiments, the one or more instructions includeone or more of the following: stop, and decrease speed.

According to various embodiments, each of the one or more traffic signalelements includes a light emitting diode.

According to various embodiments, the light emitting diode is a yellowlight emitting diode or a red light emitting diode.

According to various embodiments, the one or more image capturingdevices include a plurality of image capturing devices, and furthercomprising comparing the series of images for each image capturingdevice in the plurality of image capturing devices.

According to various embodiments, the distinct on state correlates tothe time period when the traffic signal element is in the on state forthe length of time that, in addition to equaling or exceeding the lowerthreshold, is less than or equal to an upper threshold.

According to various embodiments, the distinct off state correlates tothe time period when the traffic signal element is in the off state forthe length of time that, in addition to equaling or exceeding the lowerthreshold, is less than or equal to an upper threshold.

According to various embodiments, the one or more traffic signalelements include a plurality of redundant traffic signal elements, andthe method further includes comparing one or more cycles of each of theplurality of redundant traffic signal elements.

According to another aspect of the present disclosure, a system fordetecting a flashing light on one or more traffic signal devices isprovided. The system includes an autonomous vehicle, one or more imagecapturing devices of the autonomous vehicle configured to capture aseries of images of one or more traffic signal elements in a trafficsignal device over a length of time, and a computing device of theautonomous vehicle including a processor and a memory. The computingdevice includes instructions that cause the computing device to analyzethe series of images to determine one or more time periods at which thetraffic signal element is in an on state, and one or more time periodsat which the traffic signal element is in an off state, and analyze thetime periods to determine one or more distinct on states and one or moredistinct off states. A distinct on state correlates to a time periodwhen the traffic signal element is in the on state for a length of timethat equals to or exceeds a lower threshold, and a distinct off statecorrelates to a time period when the traffic signal element is in theoff state for a length of time that equals to or exceeds a lowerthreshold. The instructions further cause the computing device toidentify one or more cycles and, upon identifying a threshold numberadjacent cycles, classify the traffic signal element as a flashinglight. Each cycle correlates to a distinct on state immediately followedby a distinct off state, or a distinct off state immediately followed bya distinct on state.

According to various embodiments, the analyzing the series of imagesincludes generating a confidence score, for each image in the series ofimages, that the traffic signal element is in the on state.

According to various embodiments, the analyzing the series of imagesincludes generating a confidence score, for each image in the series ofimages, that the traffic signal element is in the off state.

According to various embodiments, the instructions are furtherconfigured to cause the computing device to send a command to theautonomous vehicle to perform one or more in-structions associated withthe flashing light.

According to various embodiments, the one or more instructions includeone or more of the following: stop, and decrease speed.

According to various embodiments, each of the one or more traffic signalelements includes a light emitting diode.

According to various embodiments, the light emitting diode is a yellowlight emitting diode or a red light emitting diode.

According to various embodiments, the one or more image capturingdevices include a plurality of image capturing devices, and theinstructions are further configured to cause the computing device tocom-pare the series of images for each image capturing device in theplurality of image capturing devices.

According to various embodiments, the distinct on state correlates tothe time period when the traffic signal element is in the on state forthe length of time that, in addition to equaling or exceeding the lowerthreshold, is less than or equal to an upper threshold.

According to various embodiments, the distinct off state correlates tothe time period when the traffic signal element is in the off state forthe length of time that, in addition to equaling or exceeding the lowerthreshold, is less than or equal to an upper threshold.

According to various embodiments, the one or more traffic signalelements include a plurality of redundant traffic signal elements, andthe instructions are further configured to cause the computing device tocompare one or more cycles of each of the plurality of redundant trafficsignal elements.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an example of a flashing light detection system, in accordancewith various embodiments of the present disclosure.

FIG. 2 is an example of a graphical representation of a square waverepeating pattern of a flashing light

FIG. 3 is an example of a graphical representation of a flashing yellowlight detected using a traffic signal state determination module, inaccordance with the present disclosure.

FIG. 4 is an example of a graphical representation of a beat for a redlight detected using the traffic signal state determination module, inaccordance with the present disclosure.

FIG. 5 is an example of a graphical representation of a flashing redlight detected using a traffic signal state determination module, inaccordance with the present disclosure.

FIG. 6 is a flowchart of a method for detecting flashing lights, inaccordance with the present disclosure.

FIG. 7 is an illustration of an illustrative computing device, inaccordance with the present disclosure.

DETAILED DESCRIPTION

As used in this document, the singular forms “a,” “an,” and “the”include plural references unless the context clearly dictates otherwise.Unless defined otherwise, all technical and scientific terms used hereinhave the same meanings as commonly understood by one of ordinary skillin the art. As used in this document, the term “comprising” means“including, but not limited to.” Definitions for additional terms thatare relevant to this document are included at the end of this DetailedDescription.

An “electronic device” or a “computing device” refers to a device thatincludes a processor and memory. Each device may have its own processorand/or memory, or the processor and/or memory may be shared with otherdevices as in a virtual machine or container arrangement. The memorywill contain or receive programming instructions that, when executed bythe processor, cause the electronic device to perform one or moreoperations according to the programming instructions.

The terms “memory,” “memory device,” “data store,” “data storagefacility” and the like each refer to a non-transitory device on whichcomputer-readable data, programming instructions or both are stored.Except where specifically stated otherwise, the terms “memory,” “memorydevice,” “data store,” “data storage facility” and the like are intendedto include single device embodiments, embodiments in which multiplememory devices together or collectively store a set of data orinstructions, as well as individual sectors within such devices.

The terms “processor” and “processing device” refer to a hardwarecomponent of an electronic device that is configured to executeprogramming instructions. Except where specifically stated otherwise,the singular term “processor” or “processing device” is intended toinclude both single-processing device embodiments and embodiments inwhich multiple processing devices together or collectively perform aprocess.

The term “vehicle” refers to any moving form of conveyance that iscapable of carrying either one or more human occupants and/or cargo andis powered by any form of energy. The term “vehicle” includes, but isnot limited to, cars, trucks, vans, trains, autonomous vehicles,aircraft, aerial drones and the like. An “autonomous vehicle” is avehicle having a processor, programming instructions and drivetraincomponents that are controllable by the processor without requiring ahuman operator. An autonomous vehicle may be fully autonomous in that itdoes not require a human operator for most or all driving conditions andfunctions, or it may be semi-autonomous in that a human operator may berequired in certain conditions or for certain operations, or that ahuman operator may override the vehicle's autonomous system and may takecontrol of the vehicle.

In this document, when terms such as “first” and “second” are used tomodify a noun, such use is simply intended to distinguish one item fromanother, and is not intended to require a sequential order unlessspecifically stated. In addition, terms of relative position such as“vertical” and “horizontal”, or “front” and “rear”, when used, areintended to be relative to each other and need not be absolute, and onlyrefer to one possible position of the device associated with those termsdepending on the device's orientation.

Referring now to FIG. 1 , an example of a flashing light detectionsystem 100 is provided, in accordance with various embodiments of thepresent disclosure.

According to various embodiments, the system 100 includes an autonomousvehicle 102, which includes one or more image capturing devices 104 suchas, for example, one or more cameras, configured to visually capture aplurality of images of one or more traffic signal elements 106 of one ormore traffic signal devices 108.

The system 100 includes one or more computing devices 110. According tovarious embodiments, the one or more computing devices 110 function asan AV control system. The one or more computing devices 110 can becoupled and/or integrated with the AV 102 and/or remote from the AV 102,with data collected by the one or more image capturing devices 104 beingsent, via wired and/or wireless connection, to the one or more computingdevices 110.

Traffic signal elements 106 are dynamic in that they can be changedbetween at least two states to transmit traffic instructions to one ormore drivers, and different types of traffic signal elements 106 may bepresent in a single traffic signal device 110. Examples of trafficsignal elements 106 may include, for example, a red light, a yellowlight and a green light. Other examples include lights with directionalarrows (such as arrows pointing left or right), other symbols (such as asymbol of a person walking), or words. In each of these examples, eachlight can be switched between and off state, an on state, or a flashingstate, such as a flashing yellow light and a flashing red light. Trafficsignal elements may include light emitting diodes (“LEDs”), bulbs,and/or any other suitable lighting elements.

The one or more computing devices includes a traffic signal elementstate determination module 112. The traffic signal element statedetermination module 112 is configured to analyze image data of trafficsignal elements 106, taken by the one or more image capturing devices104, in order to determine the color of the traffic signal element 106(e.g., green yellow, or red), the shape of the traffic signal element106 (e.g., circle, arrow, or lettering), and whether the traffic signalelement 106 is a solid light or a flashing light (e.g., flashing yellowor flashing red). According to various embodiments, the traffic signalelement state determination module 112 may be a Hidden Markov Model(“HMM”)-based CPU state tracker, a recurrent neural network(“RNN”)-based tracker, and/or other suitable form of traffic signalelement state determination module 112.

Many traffic signal elements 106 include an LED. LEDs typically operatevia pulse width modulation (“PWM”), in which a pulsed control signalcauses the LED to rapidly turn on and off to achieve a desired level ofbrightness. When viewing the LED with a camera, the LED can appear toflicker due to the framerate and exposure settings of the camera inconjunction with the PWM of the LED.

For an LED with a fixed duty cycle that is captured by a camera havingset framerate and exposure settings, the LED will have an oscillatingintensity that results from the beat frequency between the camera andthe LED duty cycle. The beat frequency and intensity profile may changeif any of the parameters above change, producing what is referred to asa flicker, which is the apparent changes in LED bulb brightness due tothese aforementioned effects.

Because of this, LED traffic signal elements 106 flicker. While thisflickering is often too rapid to be detectable to the human eye, adigital camera, that is part of an AV's perception system, may detectit. This flickering is unrelated to the state of the traffic signalelement 106 and should be disregarded in the state of the traffic signalelement 106. In order for the traffic signal element state determinationmodule 112 to determine whether a traffic signal element 106 isflashing, it must distinguish between a traffic signal element 106 thatis flickering (i.e., rapidly turning on and off due to PWM) and one thatis flashing (i.e., one that is being turned on and off at ahuman-perceptible speed to provide a signaling function).

In determining whether a yellow light in a traffic signal element 106 isflashing, a plurality of images of the yellow light, taken by the one ormore image capturing devices 104, are analyzed by the traffic signalstate determination module 112. The traffic signal state determinationmodule 112, for each image in the plurality of images, determineswhether the yellow light is in an on state or an off state. According tovarious embodiments, a timestamp is recorded for each image in theplurality of images, enabling the traffic signal state determinationmodule 112 to determine at which time intervals the yellow light of thetraffic signal element 106 was in the on state and at which intervalsthe yellow light of the traffic signal element 106 was in the off state.

For a flashing yellow light, the periods at which the traffic signalelement 106 is in an on state and an off state are expected to follow arepeating pattern, such as shown in the square wave of FIG. 2 .

As shown in FIG. 2 , during the yellow on state, the yellow confidenceis high, and during the yellow off state, the yellow confidence is low.The first high yellow confidence 205, corresponding to the yellow onstate, lasts for a time duration of “T₁.” The units of duration can bemilliseconds, second, and/or other suitable units of duration. In someembodiments, the units of duration are counts which indicate how manytimes the state was determined to be on or off In the square wave ofFIG. 2 , the signal is summarized as (YELLOW, T₁), (OFF, T₂), (YELLOW,T₃), (OFF, T₄), (YELLOW, T₅), etc. According to various embodiments, theon states and off states, and their respective durations, areconsolidated into a memory buffer 114.

The traffic signal state determination module 112 further analyzes thedistinct on states and off states of the yellow light of the trafficsignal element 106 to determine one or more cycles. Each cycle equatesto a pair of consecutive distinct states and their durations. Forexample, the square wave of FIG. 2 includes the following cycles:cycle1=(YELLOW, T₁)+(OFF, T₂); cycle2=(OFF, T₂)+(YELLOW, T₃);cycle3=(YELLOW, T₃)+(OFF, T₄), cycle4=(OFF, T₄)+(YELLOW, T₅), etc. Notethat consecutive cycles overlap, e.g., cycle1 and cycle2 have (OFF, T₂)in common.

According to various embodiments, in order for the traffic signal statedetermination module 112 to determine that a cycle, (state_(A),time_(A))+(state_(B), time_(B)), is considered a flashing yellow cycle(a cycle at which the yellow light of the traffic signal element 106 isflashing), it must satisfy the following conditions. The first conditionis that state_(A)=YELLOW and state_(B)=OFF, or the state_(A)=OFF andstate_(B)=YELLOW. The second condition is that the time duration of theyellow light in the on state (YELLOW) is greater than or equal to alower threshold: a minimum on state time duration (min_on_count).According to some embodiments, in addition to being greater than orequal to the minimum on state time duration (min_on_count), the secondcondition includes that the time duration of the yellow light in the onstate (YELLOW) is less than or equal to an upper threshold: a maximum onstate time duration (max_on_count). The third condition is that the timeduration of the yellow light in the off state (OFF) is greater than orequal to a lower threshold: a minimum off state time duration(min_off_count). According to some embodiments, in addition to beinggreater than or equal to the minimum off state time duration(min_off_count), the third condition includes that the time duration ofthe yellow light in the off state (OFF) is less than or equal to anupper threshold: a maximum off state time duration (max_off_count).According to some embodiments, a flashing traffic signal 106 should, inaccordance with the guidelines of the Manual on Uniform Traffic ControlDevices, have 50-60 flashing cycles per minute, with the traffic signalelement 106 being in an on state for a duration of ½ to ⅔ of each cycleduration. It is noted however, that other cycle durations and/or onstate durations may be used in conjunction with the present disclosure.

According to some embodiments, for flashing yellow lights, themin_on_count and the min_off_count are equal to 5 time duration units,and the max_on_count and the max_off_count are equal to 20 time durationunits. It is noted, however, that other suitable values for themin_on_count, max_on_count, min_off_count, and max_off_count may beused, dependent upon the characteristics of the flashing light.

The memory buffer 114 stores a list of cycles (flashing yellow cycles),which met the aforementioned conditions in a number of slots, accordingto the following rules. The first rule is that the number of flashingyellow cycles is greater than or equal to a minimum number of flashingyellow cycles required for a yellow light to be considered a flashingyellow light. The second rule is that the end location slot value of thefirst cycle is less than or equal to num_used/2, where num_used is thenumber of slots in the memory buffer 114 containing valid data.Initially, not all slots will be used, and the oldest state is held atbuffer slot 0. This condition checks that flashing pattern ishistorically present. The third rule is that the sum of the end locationslot value of the last cycle and a max_delta_last_end is greater than orequal to num_used. The “max_delta_last_end” denotes how much earlier thelast flashing yellow cycle can end. This condition checks whether arecent flashing cycle is present. According to some embodiments, themax_delta_last_end is set it at 25 for a HMM-based CPU state tracker,which corresponds to one second. It is noted, however, that othersuitable lengths of time can be used by the present system 100.

FIG. 3 shows a flashing yellow light detected using the traffic signalstate determination module 112. The x-axis denotes the number of updatesthe traffic signal state determination module 112 has returned.According to various embodiments, the traffic signal state determinationmodule 112 can return approximately 20 to 25 updates per second. It isnoted, however, that the traffic signal state determination module 112may return greater or fewer updates per second, depending upon thetraffic signal state determination module 112. The y-axis, between [0.0,1.0], denotes the confidence of the different states (the on state 305,and the off state 310). When a state 305, 310 has a confidence score of0.0, the state is determined to be not true, and when a state 305, 310has a confidence score of 1.0, the state is determined to be true.

In determining whether a red light in a traffic signal element 106 isflashing, a plurality of images of the red light, taken by the one ormore image capturing devices 104, are analyzed by the traffic signalstate determination module 112. The traffic signal state determinationmodule 112, for each image in the plurality of images, determineswhether the red light is in an on state or an off state. According tovarious embodiments, a timestamp is recorded for each image in theplurality of images, enabling the traffic signal state determinationmodule 112 to determine at which time intervals the red light of thetraffic signal element 106 was in the on state and at which intervalsthe red light of the traffic signal element 106 was in the off state.

Similar to a flashing yellow light, for a flashing red light, theperiods at which the traffic signal element 106 is in an on state and anoff state are expected to follow a repeating pattern. The remaininglogic for detecting a flashing red light is the same as the logic fordetecting a flashing yellow light, except for the bounds of themin_off_count, min_on_count, max_off_count, and max_on_count, which isdependent upon the beat characteristics of the LED light being analyzed,which causes LED flickering. The change in the bounds is to increaseinter-class variation between LED beat (for red LED lights) and thedefinition of flashing.

FIG. 4 shows LED beat for a red light detected using the traffic signalstate determination module 112. The x-axis denotes the number of updatesthe traffic signal state determination module 112 has returned. They-axis, between [0.0, 1.0], denotes the confidence of the differentstates (the on state 405, and the off state 410), for the red LED light.When a state 405, 410 has a confidence score of 0.0, the state isdetermined to be not true, and when a state 405, 410 has a confidencescore of 1.0, the state is determined to be true.

In the initial region between 0 and 200, the states 405, 410 and theirdurations are (RED, 10), (OFF, 24), (RED, 11), (OFF, 24), (RED, 12),(OFF, 25), and (RED, 16). If the max_off_count for red light is set to20, the limit is close to the duration of the OFF count. This wouldresult is the system 100 falsely determining that solid red LED havingbeat is a flashing red light. Compare the red LED beat of FIG. 4 withthe flashing red LED of FIG. 5 . In FIG. 5 , the states 405, 410 andtheir durations form a distinct on state/off state pattern that is notpresent in the beat.

According to various embodiments, a traffic signal element 106 of atraffic signal device 108 is only labeled as flashing if it meets theaforementioned conditions described herein. If a traffic signal element106 in a traffic signal device 108 is not determined to be flashingbased on the input, the state of the traffic signal element 106 islabeled as not flashing.

According to various embodiments, the system 100 includes a plurality ofimage capturing devices 104. The aforementioned tests for determiningwhether a traffic signal element 106 is flashing can be performed forimage data collected from each of the plurality of image capturingdevices 104. The results for each of the image capturing devices 104 canthen be compared for validating the results for one or more imagecapturing devices 104.

A traffic signal device 108 may include a plurality of traffic signalelements 106. According to various embodiments, each traffic signalelement 106 of the traffic signal device 108, or each of a subset oftraffic signal elements 106 of the traffic signal device 108, areindividually classified as being flashing or not flashing. Thisclassification may be performed using image data from each camera 104for a particular traffic signal element 106. For example, the periods ofoff and on for faces viewed by each of multiple different cameras 104should be consistent with one another. According to various embodiments,an intensity profile for each traffic signal element 106 may also bestored, in addition to the classification data.

Traffic signal devices 108 are often redundant and convey repeatedinformation. For example, a traffic signal device 108 may include aplurality of flashing lights (e.g., a plurality of flashing red lights,a plurality of flashing yellow lights, etc.). Traffic signal elements106 of the same type and flash settings (e.g., a pair of flashing yellowlights, a pair of flashing red lights, etc.) should have consistentflashing classifications. For example, if a first traffic signal element106 and a second traffic signal element 106 are redundant, then thefirst and second traffic signal element 106 should be flashing at thesame time in order for the set of traffic signal elements 106 to beconsidered flashing.

An intersection may include a plurality of traffic signal devices 108.If two or more of the traffic signal devices 108 in the plurality oftraffic signal devices 108 include redundant flashing traffic signalelements 106, the flashing traffic signal elements 106 of one trafficsignal device 108 should be in phase with the flashing traffic signalelements 106 of the other traffic signal devices 108.

Accurately determining whether a state of a traffic signal element 106is solid or flashing, and accurately differentiating between aflickering a flashing LED light, increases the likelihood that the AV's102 movements following the instructions of the traffic signal element106. This, in turn, increases ride safety and rideenjoyment/satisfaction, thus improving upon the existing technologies.

Referring now to FIG. 6 , a flowchart of a method 600 for detecting aflashing light on a traffic signal device is illustratively depicted.

According to various embodiments, at 605, one or more image capturingdevices coupled to one or more AVs each capture a series of images of atraffic signal element of a traffic signal device over a length of time.The traffic signal element may include an LED such as, for example, ayellow LED or a red LED.

At 610, the series of images, are analyzed to determine one or more timeperiods at which the traffic signal element is in an on state, and oneor more time periods at which the traffic signal element is in an offstate. According to various embodiments, the analyzing the series ofimages includes generating a confidence score, for each image in theseries of images, that the traffic signal element is in the on stateand/or that the traffic signal element is in an off state. The timeperiods, at 615, are then analyzed in order to determine one or moredistinct on states and one or more distinct off states of the trafficsignal element as described herein.

At 620, one or more cycles are identified. Each cycle correlates to adistinct on state immediately followed by a distinct off state, or adistinct off state immediately followed by a distinct on state.According to various embodiments, the one or more image capturingdevices include a plurality of image capturing devices, each imagecapturing device in the plurality of image capturing devices capturing aseries of images of the traffic signal element over the length of time.At 625, the series of images for each image capturing device in theplurality of image capturing devices are compared, validating the dataof the one or more of the image capturing devices in the plurality ofimage capturing devices. According to various embodiments, one or moretraffic signal devices may include two or more redundant flashinglights. The two or more redundant flashing lights, at 630, are comparedto determine if the two or more redundant flashing lights have matchingon state and off state characteristics.

The cycles are analyzed to determine one or more groupings of adjacentcycles. Upon identifying a threshold number of adjacent cycles, thetraffic signal element, at 635, is classified as a flashing light.According to various embodiments, in the event that that there are aplurality of redundant traffic signal elements, in addition toidentifying a threshold number of adjacent cycles, in order to beclassified as a flashing light, the plurality of redundant trafficsignal elements must be determined to have matching on state and offstate characteristics.

One the flashing light or lights have been classified, a command, at640, is then sent to the AV to perform one or more instructionsassociated with the flashing light. For example, the instructions caninclude instructing the AV to stop, slow down (i.e., decrease speed),and/or other suitable instructions.

Referring now to FIG. 7 , an illustration of an illustrativearchitecture for a computing device 700 is provided. The computingdevice 110 of FIG. 1 is the same as or similar to computing device 700.As such, the discussion of computing device 700 is sufficient forunderstanding the computing device 110 of FIG. 1 .

Computing device 700 may include more or less components than thoseshown in FIG. 6 . However, the components shown are sufficient todisclose an illustrative solution implementing the present solution. Thehardware architecture of FIG. 7 represents one implementation of arepresentative computing device configured to one or more juke events,as described herein. As such, the computing device 700 of FIG. 7implements at least a portion of the method(s) described herein.

Some or all components of the computing device 700 can be implemented ashardware, software and/or a combination of hardware and software. Thehardware includes, but is not limited to, one or more electroniccircuits. The electronic circuits can include, but are not limited to,passive components (e.g., resistors and capacitors) and/or activecomponents (e.g., amplifiers and/or microprocessors). The passive and/oractive components can be adapted to, arranged to and/or programmed toperform one or more of the methodologies, procedures, or functionsdescribed herein.

As shown in FIG. 7 , the computing device 700 comprises a user interface702, a Central Processing Unit (“CPU”) 706, a system bus 710, a memory712 connected to and accessible by other portions of computing device700 through system bus 710, a system interface 760, and hardwareentities 714 connected to system bus 710. The user interface can includeinput devices and output devices, which facilitate user-softwareinteractions for controlling operations of the computing device 700. Theinput devices include, but are not limited to, a physical and/or touchkeyboard 750. The input devices can be connected to the computing device700 via a wired or wireless connection (e.g., a Bluetooth® connection).The output devices include, but are not limited to, a speaker 752, adisplay 754, and/or light emitting diodes 756. System interface 760 isconfigured to facilitate wired or wireless communications to and fromexternal devices (e.g., network nodes such as access points, etc.).

At least some of the hardware entities 714 perform actions involvingaccess to and use of memory 712, which can be a random access memory(“RAM”), a disk drive, flash memory, a compact disc read only memory(“CD-ROM”) and/or another hardware device that is capable of storinginstructions and data. Hardware entities 714 can include a disk driveunit 716 comprising a computer-readable storage medium 718 on which isstored one or more sets of instructions 720 (e.g., software code)configured to implement one or more of the methodologies, procedures, orfunctions described herein. The instructions 720 can also reside,completely or at least partially, within the memory 712 and/or withinthe CPU 706 during execution thereof by the computing device 700. Thememory 712 and the CPU 706 also can constitute machine-readable media.The term “machine-readable media”, as used here, refers to a singlemedium or multiple media (e.g., a centralized or distributed database,and/or associated caches and servers) that store the one or more sets ofinstructions 720. The term “machine-readable media”, as used here, alsorefers to any medium that is capable of storing, encoding or carrying aset of instructions 720 for execution by the computing device 700 andthat cause the computing device 700 to perform any one or more of themethodologies of the present disclosure.

Although the present solution has been illustrated and described withrespect to one or more implementations, equivalent alterations andmodifications will occur to others skilled in the art upon the readingand understanding of this specification and the annexed drawings. Inaddition, while a particular feature of the present solution may havebeen disclosed with respect to only one of several implementations, suchfeature may be combined with one or more other features of the otherimplementations as may be desired and advantageous for any given orparticular application. Thus, the breadth and scope of the presentsolution should not be limited by any of the above describedembodiments. Rather, the scope of the present solution should be definedin accordance with the following claims and their equivalents.

What is claimed is:
 1. A method of detecting a solid light of at least one traffic signal element, the method comprising: by one or more image capturing devices of an autonomous vehicle, capturing a series of images of at least one traffic signal element over a length of time; by an autonomous vehicle control system of the autonomous vehicle: analyzing the series of images to determine time intervals when the traffic signal element is in an on state and time intervals when the traffic signal element is in an off state; further analyzing the series of images to determine one or more flashing cycles comprising a first time interval when the traffic signal element is in a first state for at least a first minimum duration and less than a first maximum duration followed by a second time interval when the traffic signal element is in a second state for at least a second minimum duration threshold and less than a second maximum duration, the first state different than the second state; determining an end time of a last flashing cycle in the series of images; and upon detecting a minimum number of flashing cycles in the series of images and determining that a delta time between the end time of the last flashing cycle and the end time of the series of images is greater than a maximum time delta, classifying the traffic signal element as a solid light.
 2. The method of claim 1, wherein the analyzing the series of images includes generating a confidence score, for each image in the series of images, that the traffic signal element is in the on state.
 3. The method of claim 1, wherein the analyzing the series of images includes generating a confidence score, for each image in the series of images, that the traffic signal element is in the off state.
 4. The method of claim 1, further comprising, by the autonomous vehicle control system of the autonomous vehicle, sending a command to the autonomous vehicle to perform one or more instructions associated with the solid light.
 5. The method of claim 4, wherein the one or more instructions include one or more of the following: stop; and decrease speed.
 6. The method of claim 1, wherein the traffic signal element includes a light emitting diode.
 7. The method of claim 6, wherein the light emitting diode is a yellow light emitting diode or a red light emitting diode.
 8. The method of claim 1, wherein the one or more image capturing devices include a plurality of image capturing devices, and further comprising comparing the series of images for each image capturing device in the plurality of image capturing devices.
 9. The method of claim 1, wherein the traffic signal element includes a plurality of redundant traffic signal elements, and further comprising comparing one or more cycles of each of the plurality of redundant traffic signal elements.
 10. A system for detecting a solid light of at least one traffic signal element, the system comprising: one or more image capturing devices of an autonomous vehicle, each image capturing device being configured to capture a series of images of at least one traffic signal element over a length of time; and a computing device of the autonomous vehicle, including: a processor; and a memory, wherein the computing device includes instructions, wherein the instructions cause the computing device to: analyze the series of images to determine time intervals when the traffic signal element is in an on state and time intervals when the traffic signal element is in an off state; further analyze the series of images to determine one or more flashing cycles comprising a first time interval when the traffic signal element is in a first state for at least a first minimum duration and less than a first maximum duration followed by a second time interval when the traffic signal element is in a second state for at least a second minimum duration threshold and less than a second maximum duration, the first state different than the second state; determine an end time of a last flashing cycle in the series of images; and upon detecting a minimum number of flashing cycles in the series of images and determining that a delta time between the end time of the last flashing cycle and the end time of the series of images is greater than a maximum time delta, classifying the traffic signal element as a solid light.
 11. The system of claim 10, wherein the analyzing the series of images includes generating a confidence score, for each image in the series of images, that the traffic signal element is in the on state.
 12. The system of claim 10, wherein the analyzing the series of images includes generating a confidence score, for each image in the series of images, that the traffic signal element is in the off state.
 13. The system of claim 10, wherein the instructions are further configured to cause the computing device to send a command to the autonomous vehicle to perform one or more instructions associated with the solid light.
 14. The system of claim 13, wherein the one or more instructions include one or more of the following: stop; and decrease speed.
 15. The system of claim 10, wherein each of the one or more traffic signal elements includes a light emitting diode.
 16. The system of claim 15, wherein the light emitting diode is a yellow light emitting diode or a red light emitting diode.
 17. The system of claim 10, wherein the one or more image capturing devices include a plurality of image capturing devices, and wherein the instructions are further configured to cause the computing device to compare the series of images for each image capturing device in the plurality of image capturing devices.
 18. The system of claim 10, wherein the one or more traffic signal elements include a plurality of redundant traffic signal elements, and wherein the instructions are further configured to cause the computing device to compare one or more cycles of each of the plurality of redundant traffic signal elements. 